Mutation testing in the wild: findings from GitHub

Abstract Mutation testing exploits artificial faults to measure the adequacy of test suites and guide their improvement. It has become an extremely popular technique as evidenced by vast literature, numerous tools, research events on topic. Previous survey papers have successfully compiled state research, its evolution, problems, challenges. However, use mutation in practice is still largely unexplored. In this paper, we report results a thorough study GitHub projects. Specifically, first performed search for 127 total, automatically searched repositories including evidence use. Then, focused top ten most widely used based previous results, manually revised classified over 3.5K active importing them. Among other findings, observed recent upturn interest activity, with Infection (PHP), PIT (Java) Humbug (PHP) being tools years. The predominant development, followed teaching learning, projects, although significant differences among found literature—less adopted research—and those only—more more development. Our work provides new encouraging perspective testing.